Extensive eigenvalues in spin-spin correlations: a tool for counting pure states in Ising 

spin glasses 
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We study the nature of the broken ergodicity in the low temperature phase of Ising spin glass 
systems, using as a diagnostic tool the spectrum of eigenvalues of the spin-spin correlation function. 
We show that multiple extensive eigenvalues of the correlation matrix dj = (SiSj) occur if and 
only if there is replica symmetry breaking. We support our arguments with Exchange Monte-Carlo 
results for the infinite-range problem. Here we find multiple extensive eigenvalues in the RSB phase 
for TV > 200, but only a single extensive eigenvalue for phases with long-range order but no RSB. 
Numerical results for the short range model in four spatial dimensions, for TV < 1296, are consistent 
with the presence of a single extensive eigenvalue, with the subdominant eigenvalue behaving in 
agreement with expectations derived from the droplet model. Because of the small system sizes 
we cannot exclude the possibility of replica symmetry breaking with finite size corrections in this 
regime. 
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I. INTRODUCTION 

In three spatial dimensions or higher it is now accepted 
that Ising spin systems with random exchange interac- 
tions, exhibiting disorder and frustration, undergo a tran- 
sition from a paramagnetic phase to a glass phase at a 
finite temperature.El By contrast, the nature of the glass 
phase at finite dimensions is still a subject of much de- 
bate in the literature, with two main competing points 
of view. One follows Pacisi's solution of the Sherrington- 
Kirkpatrick (SK) modelo using a replica symmetry break- 
ing (RSB) ansatzll The Parisi solution involves bro- 
ken ergodicity of a subtler form than that found in a 
conventional ferromagnet: configuration space is broken 
into many ergodic regions, separated by energy barri- 
ers which diverge in the thermodynamic limit. Most 
of these regions — which we will also call pure states — 
are unrelated to one another by any symmetry of the 
Hamiltonian. However, in the case of a Hamiltonian 
with global spin inversion symmetry, each of these re- 
gions has an associated region related to it by global 
spin inversion, the pair forming together what we will 
call a pure state pair (PSP). This RSB picture is almost 
certainly correct in the limit of infinite spatial dimen- 
sion; and it has been argued that the RSB picture also 
applies, more or less unchanged, to the frozen phase of 
finite-dimensional spin glasses. The other point of view 
is the 'droplet' picturejj which, in sharp contrast to the 
picture just described, postulates the presence of a sin- 
gle PSP in the low-temperature phase. In this paper we 
focus on the fundamental difference between these two 
pictures, namely, the nature of the ergodicity breaking 
in spin glasses — or, put more simply, the number of 'val- 



leys' (ergodic regions) in the low-temperature phase. 

The glass transition is characterized by the Edwards- 
Anderson spin-glass order parameter 



(1) 



which becomes nonzero [i.e., of 0(1)] for T < T c . Here 
Si = ±1 is the spin at site i, TV is the number of spins in 
the system, [ ] av indicates an average over disorder real- 
izations and ( ) indicates a thermal average. The order 
parameter q may also be obtained as the first moment of 
the overlap distribution function 



a/3 



(2) 



where a,/3 label pure states, q af3 = J2i( s i) a ( s i) P / N in- 
dicates the overlap between the local magnetizations in 
two pure states, and w a is the thermodynamic weight of 
pure state a. In the thermodynamic limit, P(q) is pre- 
dicted to have very different behaviors in the two pictures 
mentioned above. For a single PSP, P{q) approaches a 
pair of delta functions at ±g mQX , where q max is the self- 
overlap of each pure state in the pair, and —q m ax is the 
overlap between one pure state and the other. For the 
diverging (countable) number of PSPs in the SK problem 
P(q) is nonzero over a finite interval 

Qmax 

(this property depends essentially on the fact that P(q) 
is a disorder averaged quantity). So far, the main tool 
for detecting non-trivial broken ergodicity in finite di- 
mension at nonzero temperatures has been the scaling 
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of the overlap distribution function P(q) at q — as a 
function of system size: for a single PSP it must scale 
to zero, while it remains nonzero in the thermodynamic 
limit if there is RSB. Several numerical studies have sug- 
gested a behavior in fwite dimensions similar to the one 
at the mean field levelBB Other studies using the Migdal- 
Kadanoff approximation^ and still others investigating 
the ground state susceptibility to boundary conditions!!! 
have suggested otherwise and favor the droplet model. 
None of these studies has conclusively resolved which 
type of broken ergodicity takes place in the low temper- 
ature phase in finite dimensions, motivating a search for 
new approaches. 

In this paper we propose and apply a rather direct 
method for determining the number of PSPs for an Ising 
spin glass. n Our main ideas were introduced in a previ- 
ous Letterja here we provide an extended development 
of these ideas, along with further analytical and nu- 
merical results. Our method involves the study of the 
spectral properties of the spin-spin correlation_function 
(SiSj) = Cij. It is inspired by Yang's analysisEj of cor- 
relation functions to detect off-diagonal long-range order 
(ODLRO) in superfluids. In the case of superfiuids, Yang 
showed that the existence of ODLRO is equivalent to the 
presence of an extensive eigenvalue in the spectrum of 
the one-particle density matrix. We argue that, in the 
case of classical spin glasses, the spectrum of the spin- 
spin correlation function contains a distinct signature 
which allows one to determine unambiguously whether 
or not there are many pure state pairs, i.e. whether or 
not RSB occurs. First of all, it is clear that the pres- 
ence of at least one extensive eigenvalue signals the pres- 
ence of long-range order. What we further show is that 
the number of extensive eigenvalues determines unam- 
biguously the number of pure state pairs: the spectrum 
contains exactly one extensive eigenvalue if and only if 
there is exactly one pure state pair, and it contains more 
than one extensive eigenvalues if and only if there are 
more than one pure state pairs. We also argue that the 
extensive eigenvalues dominate the trace of C, and that 
the nonextensive eigenvalues scale with the number TV 
of spins in the system with a power lower than 1. We 
confir|Hi-these arguments by performing Exchange Monte 
CarloOiia (EMC) simulations for the SK model in various 
regimes, for which we know the nature of the ergodic- 
ity breaking in the equilibrium state. That is, we find 
multiple extensive eigenvalues in the spin-glass phase, 
but only a single such eigenvalue in the ferromagnetic 
phase and in the paramagnetic phase in a field. In the 
RSB phase (where one expects many PSPs) we find that, 
for the range of system sizes that we have studied, the 
eigenvalue spectrum is dominated by a small number of 
extensive eigenvalues. Making the simplifying assump- 
tion that the thermal average is dominated by only two 
PSPs, we are able to introduce an analytically calculable 
model which reproduces the eigenvalue spectrum for the 
SK model surprisingly well. Finally, we use EMC simu- 
lations to study the Edwards-Anderson (near-neighbor) 



model in four spatial dimensions. Our results, for system 
sizes 3 4 < N < 6 4 , are compatible with the presence of 
only one extensive eigenvalue of C. The second largest 
eigenvalue of C is very well fit (with x 2 = 0.132) by 
a power-law scaling with N, with an exponent (smaller 
than one) that is consistent with the value predicted by 
the droplet theory. An alternative fit with a form that 
allows for an extensijze piece in the second largest eigen- 
value of C is possible^, although it would imply that the 
ratio of the eigenvalue over N saturates to a finite value, 
something that is not observed in our data. 

We organize this paper as follows. In Sec. || we show 
how the properties of the spectrum of Cij below T c can 
be used to distinguish between RSB and a single PSP 
in the low temperature phase. In Sec. [II we discuss 



numerical results for the SK model in its various regimes. 
In Sec. IV we introduce a two-PSP model and show that 



it gives results much like the numerical results for the SK 
model. In Sec. we present and discuss the results of our 
EMC simulations for the short-range Edwards- Anderson 
model in four dimensions. Finally, in Sec. VI we present 
our conclusions. 



II. SPIN-SPIN CORRELATORS AND PURE 
STATES IN ISING SYSTEMS 

Guided by the analogy to Yang's analysis of the matrix 
elements of the one particle density matrix in a superfluid 
system, we study the spectrum {A»}i=i ... jy of the spin- 
spin correlation matrix Cij of an Ising spin system. This 
matrix has an extensive trace TrC = 'Yl/ASf) — N. In 
addition, it is positive semi-definite: for an arbitrary real 
N-dimensional vector \v), 

(v\C\v) = ViCijVj = ^Vi (StSj) Vj 

ij ij 

= <(!><Si) a >>o. 



Above the ordering temperature Cy decays exponentially 
to zero at large ry, so it has no large eigenvalues. For 
T — > oo (or for T > T c in the SK model, for which dis- 
tance is not meaningful), the correlation matrix reduces 
to the identity Cy = Sij, and consequently all its eigen- 
values are equal to one. Below T c , however, Cij is nonzero 
almost everywhere due to the ordering of the spins. For 
one PSP, in the low temperature limit T — > 0, Cij — ► ±1, 
C has one eigenvalue which approaches N as T — > 0, and 
the rest of the eigenvalues go to zero. Hence, there is 
a transition in the distribution of the eigenvalues of Cy 
as T crosses T c . Specifically, just as in the case of su- 
perfluids and fcrromagnets and antiferromagnets, we can 
detect the existence of long range order below T c by the 
presence of one (or more) extensive eigenvalue (s). This 
approach based on examining the spectrum of the cor- 
relation functions has a definite advantage in the study 
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of disordered systems: we can eliminate the necessity for 
guessing the nature of the order — i.e., the eigenvectors 
corresponding to the extensive eigenvalues — and still de- 
tect its existence. We will also find that this spectrum 
can give unique and clear information about the number 
of pure states in a frozen, disordered phase. 

Although the spin-spin correlation function dj has not 
been extensively used to probe the nature of the broken 
ergodicity in the spin glass phase, many related quantities 
have been used to study some of the static properties of 
spin glasses. For example, the quantity 

= [ Tr C 2 /N% v (3) 

is a commonly used order parameter in spin-glass sys- 
tems.!^ Also, the spin-glass susceptibility Xsg f° r one dis- 
order realization (i.e. for a fixed set of spin couplings Jij) 

XsG = ^E((^)-(5 i )(5 j )) 2 (4) 

ij 

is given, for T > T c , by 

X J SG-^J2 C ^ T " C2 / N - 

ij 

We see that the disorder average of C 2 contains in- 
teresting information about the freezing of the spins 
(while in contrast the disorder average of C contains no 
information — it is just the unit matrix). For T < T c , 
becomes of order O(l), the largest eigenvalue Ai of C be- 
comes of order N, and Tr C 2 = J^. becomes of order 
A 2 . Hence our measure of order (Ai ~ N) is consistent 
with earlier measures used for spin glasses. 

We wish further to obtain new and independent in- 
formation from the spectrum of C; towards this goal, 
in the following two subsections we will argue that one 
can detect the presence or absence of order with many 
valleys (i.e., RSB) simply by counting the number of ex- 
tensive eigenvalues of C. To make this connection, the 
main idea used will be.-that pure states are character- 
ized by their clustering^ property i.e. that the spin- 
spin correlation function at long distances can be approx- 
imately decomposed as a linear combination of (possibly 
non-orthogonal) projectors onto the subspaces associated 
with the pure states present in the system, i.e.: 

(S^) « 5> a < 5 <> a < 5 *>°> (6) 

a 

where ( ) a denotes a thermal average restricted to the 
pure state a. From this relation, a connection will be 
obtained between the number of extensive eigenvalues of 
the left hand side and the number of pure state pairs 
present in the right hand side. The rest of this section 
is devoted to deriving this connection, and to estimating 
the effects on the spectrum due to the terms neglected in 
Eq. (§). 



A. Single pure state pair 

We first consider the case of a single PSP. We show 
that in this case there is only one extensive eigenvalue 
that dominates the trace of C. 

Without loss of generality, we can consider the case of 
only one pure state. This applies directly if one of the 
pure states in the pair is selected by an external field 
or a boundary condition. But even when the two pure 
states are present, the spin-spin correlation matrix for the 
system is the same as if only one of them was present, 
simply because ij— involves a product of an even number 
of spin variables O 

Hence we can rewrite Cij as 

dj = C% + Vij (7) 
C°. = {5 i )(S i ) (8) 
Vij = {SiS s ) c , (9) 

where (SiSj) c = (SiSj) — (Si)(Sj) is the connected cor- 
relation function. By arguments similar to the one used 
to show that C is positive semi-definite, both C° and V 
are positive semi-definite (see Appendix . 

Let's ignore for the moment the connected part V, and 
concentrate on the matrix C . This matrix is propor- 
tional to the projector onto the vector Sj = {Si}, thus it 
has exactly one extensive eigenvalue 

«i = TrC° = {s\s) = J2^ 2 = N V U ( 10 ) 

i 

(where g 11 is the self-overlap of the pure state) with 
eigenvector proportional to Si, and N — 1 eigenvectors 
with eigenvalue zero. 

We then ask how the matrix V changes this distribu- 
tion of-,eigenvalues. By the clustering property of pure 
statestJ, the typical element of V vanishes as N — > oo; 
hence it is reasonable to assume that the typical value of 
the ratio between the largest eigenvalue v of V and the 
largest eigenvalue k\ of C° goes to zero in the thermo- 
dynamic limit. By a detailed variational argument (see 
Appendix ^) , it can be shown that the largest eigenvalue 
Ai of C is bounded between K\ and k,\ + v, and in the 
case that d « ki the second largest eigenvalue A2 of C 
is bounded above by v times a number that goes to one. 
In other words, the largest eigenvalue of the correlation 
matrix remains extensive (and, in fact, it changes very 
little) when the effect of V is included. This result can 
also be recovered more intuitively by applying pertur- 
bation theory to the problem of estimating the effect of 
V. 

The problem that remains to be solved is, therefore, 
the estimation of the largest eigenvalue of V. A possi- 
ble assumption would be that the off-diagonal parts of V 
have a typical behavior in the large- A limit of ~ N~ s , 
with S > 0. (The diagonal elements are always ~ N°, 
but they have an effect of order N° on the eigenvalues.) 
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The largest eigenvalue of V, and therefore the second 
largest eigenvalue of C, are then of order N 1 ^ 5 . One can 
also view this result in the following way. V can at most 
reweight the eigenvalues of C as if it is an additional pure 
state with thermodynamic weight cx N~ s (see Sec. fV\) , in 
which case it gives rise to an eigenvalue (again) cx N l ~ s . 
An eigenvalue of order N x ~~ s is, in principle, distinguish- 
able from an eigenvalue of order N, since 8 > 0. 

In some cases it is possible to obtain a stronger bound 
on the decay exponent 8. For instance, if we assume 
that xsg ~ N°, then we get 8 > 1/2 by simple power 
counting. This should be the-case everywhere above the. 
Almeida-Thouless (AT) lineEj in the phase diagran£3 
for the SK model. Above this line there is a param- 
agnetic phase with a single pure state, with long-range 
order trivially induced when the external magnetic field 
h is nonzero. There is also a conventional ferromagnetic 
phase with a single PSP, in another part of the phase di- 
agram but still above the AT instability. Either of these 
phases should have Vij decaying with 8 > 1/2. For a pure 
ferromagnet with no frustration or disorder, assuming 
(as appropriate for a single PSP) the uniform suscepti- 
bility x = (1/^0 J2ij( s i S 3)c is of 0(1) gives the stronger 
constraint 8 = 1. We expect this latter limit to be ap- 
proached for the SK problem, when either the external 
magnetic field h or the average ferromagnetic coupling 
Jo are sufficiently large. 

While the above arguments make no direct reference 
to spatial dimension, they do rely on the notion of a 
'typical' element of V. This idea is certainly appropri- 
ate for infinite-range models such as the SK problem; 
and the above arguments may also be applied to any 
finite-dimensional problem for which a typical behavior 
of Vij can be defined. For example, one can define a typ- 
ical Vij — (SiSj) c for any magnetic system with a finite 
correlation length; here the typical Vij is exponentially 
small and so the matrix V does not have significant ef- 
fects on the eigenvalues of C . However, correlations in 
finite-dimensional spin glasses, in the frozen phase* are 
thought to fall off more slowly than exponentiallyjlj giv- 
ing an. iofinite spin-glass susceptibility in the spin-glass 
phasaj'cZI and rendering the notion of a typical element 
of V problematic. n 

Hence we examine carefully the 'droplet' theory,cl 
which is the outstanding candidate for a theory of finite- 
dimensional spin glasses without RSB. In this theory the 
low energy excitations at large distances are assumed to 
be large droplets of collectively overturned spins of size 
L, whose energy scales as L e . It follows that the major- 
ity of the elements of Vij are exponentially small; how- 
ever there is also a set of 'big' elements which are of 
0(1) in magnitude. These elements occur when i and j 
lie within the same 'active' (coherently flipping) droplet; 
this makes (Si) and (Sj) small, while leaving (SiSj) and 
(SiSj) c large. The fraction of these big elements is of or- 
der l/L 9 , where L is the system size and 8 is a scaling ex- 
ponent from the zero-temperature fixed point. Although 
this 'big' fraction vanishes in the thermodynamic limit, 



it still can have large effects at large N. For example, 
let us suppose that a finite sample of size L is dominated 
by one large active droplet of size of order L. In such 
a case the big elements of V , appearing with probability 
~ l/L 6 = l/N 9 / d , are coherent, so that the largest eigen- 
value of V is of order N x N- / d = N l - 9 ' d . (We have 
verified this with simple numerical experiments.) Given 
this bound on the eigenvalues of V, the largest eigen- 
value of V is again much smaller than the largest eigen- 
value of C°, and again the second largest eigenvalue A2 
of C — C° + V is of order 7V 1_e / d or smaller. Hence one 
gets a decay exponent 82 for A2/-/V equal to 6/d. Given 
the assumptions leading to this conclusion, this value is a 
lower bound for the rate of decay of X2/N. It is plausible 
however that, for a range of system size N = L d not too 
large, the assumption of dominance by a single active 
droplet can hold, giving this lower bound for 8%. Fur- 
thermore, the numerical value of the latter can be-quite 
small: about 0.19/3 ~ 0.0&3-,in three dimensions ,113 and 
about 0.7/4 0.17 in four .Ell Hence one needs good nu- 
merical data for A2 (N) in a finite-dimensional spin glass 
to distinguish RSB (with 82 = 0) from a X2/N which is 
weakly decaying due to droplet excitations. 

B. Many pure state pairs 

In the previous subsection we have presented strong 
arguments, assuming there is only a single PSP in the 
low temperature phase, that there can be only one ex- 
tensive eigenvalue of C. It follows from our argument 
that the observation of more than one extensive eigen- 
value directly implies RSB. In short, letting Npsp be the 
number of PSPs, and Neev be the number of extensive 
eigenvalues, we found 

(Npsp = 1) => (Neev = 1) (11) 

or 

(Neev > 1) (Npsp > 1) = (RSB). (12) 

Now we would like to argue that the converse is also valid, 
i.e., to find some necessary consequence of RSB in the 
eigenvalue spectrum. Hence we will assume RSB (that 
there is more than one pure state pair present), and then 
determine how many extensive eigenvalues there should 
be in the spectrum of the spin-spin correlation matrix. 

Let us suppose that there are p > 1 pure states, char- 
acterized by the magnetizations {mf } (mf = (Si) a ), and 
thermodynamic weights {w a }, with a — 1, • ■ • ,p. Here 
we only include pure states whose thermodynamic weight 
w a is nonvanishing. We now decompose the correlation 
function into the contributions coming from each pure 
state: 

C ij = J2^ a {SiS j ) a (13) 

ct=l 

= C% + Vij , (14) 
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with 



G % - E 



w a mfmf 
l j 



a=l 



Vij = E ^ 



V5 = (SiS j )?. 



(15) 

(16) 
(17) 



By the clustering property of pure states, we may assume 
that V is small compared with C° . As in the case of one 
PSP, we will proceed in two steps. First we will study 
the spectrum of C°, and later we will include the effect 
of V. 

Let's define the p vectors {\<fir)}r=i,—,p 



~)j - E C /5 A 
/3=1 



~~§ I 3 

nPrri" - 



(18) 



with the coefficients to be determined later. For an 
appropriate choice of the Cg, these vectors can be shown 
to be eigenvectors of C°-. In fact, 

N N p 

3=1 3=1 a,P=l 

= Nj2V^m?J2A aP c r p , (19) 

a=l /3=1 

where we have defined the real, symmetric p x p matrix 
A a/ 3 = \/w a q a ^V w@ . The matrix A a p has p orthonor- 
mal eigenvectors {c^} with eigenvalues {u r } r =i r .. jP . By 
inserting one of these eigenvectors in Eq. (|l8|) we obtain 



N 

E/ " ' * 

3=1 



C%{<t>r)j = Na r (4> r ) t 



(20) 



Thus, for each nonzero eigenvalue a r of A, an eigenvector 
of C° is obtained with eigenvalue n r — Na r . Let's denote 
by p the number of linearly independent magnetization 
vectors among the magnetizations of the p pure states 
(1 < P < p)- This number need not be equal to p: for 
example, the magnetizations of the two pure states in a 
PSP are not linearly independent, since one of them is 
— 1 times the other. Having p — 1 is equivalent to saying 
that there is either only one pure state present, or there is 
exactly one PSP. Therefore, for the droplet picture p = 1 
and for the RSB picture p > 1 strictly, i.e.: 



(Npsp > 1) => (p > 1). 



(21) 



It is an exercise in linear algebra to show that (see Ap- 
pendix |Aj) : (i) the number of nonzero eigenvalues of A 
is exactly p, (ii) all of these nonzero eigenvalues are pos- 
itive, (iii) the corresponding eigenvectors of C° are lin- 
early independent, and (iv) the remaining N — p linearly 



independent eigenvectors of C° have zero eigenvalue and 
are orthogonal to all pure state magnetizations. As a 
consequence, the number of nonzero extensive eigenval- 
ues of C° is equal to p, the number of pure states with 
linearly independent magnetizations. 

Next we assess the effects of Vij = ^ a w a V°- . We 
assume that the sum over pure states is finite (see below, 
and Ref. Hence, even if the VS decay at different 
rates with N, we can still take the typical element of Vij 
to decay with TV at least as fast as N~ s for some S > 0. 
Hence the largest eigenvalue of V is of order N 1 ^ 6 or 
smaller. From this we can show (Appendix |b|) that there 
are still p extensive eigenvalues for C = C° + V , 

N EEV > P- (22) 

Combining this with Eq. ( pl| ) it follows that 

{Npsp > 1) => (AW > I)- (23) 



that is, the converse of Eqs. (11) and (|12|). 

It is plausible, although not proven, that in general a 
complete set of pure states not related by spin inversion 
(thus constituting one-half of the total set of pure states) 
will all be, with probability one, linearly independent, so 
that p is just the number of PSPs, i.e. AflSP = Neev- 

Note that, in the Parisi RSB solutionis to the mean- 
field problem, the number of PSPs grows with N, at 
a rate which is not knownj—.The ultrametric structure 
of these pure states impliesEil that they cannot grow in 
number faster than N. However there is a stronger con- 
straint, coming from Eq. (47) of Ref. which states 

that E Q ( W ") 2 = CK 1 )- This tells us that the diverging 
number of pure states do not have equal thermodynamic 
weight; instead a finite number of them dominate the 
sum of the weights w a , with the rest having negligible 
weight. 

One might ask whether the ultrametric structure of 
the space of pure states might imply some constraint on 
the number which are linearly independent. However we 
find no such constraint in general. For example, one can 
construct, for any number n < N (where N is the di- 
mension of the vector space), a set of n vectors which 
are both linearly independent and ultrametric; but one 
can also construct a set which is linearly dependent and 
ultrametric. 

It may be possible to derive tighter bounds on these 
quantities via further theoretical work. In the following 
section we provide some further information, obtained 
from equilibrium Monte-Carlo studies, on Neev I0r a 
finite range of N in the SK problem. 



III. RESULTS FOR THE SK MODEL 



The SK model is described by the Hamiltonian 



H — — ^ Jij SiSj h ^ Si , 

ijtj i 



(24) 
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with [J y -] av = J /(N- 1) and [J?-] av - [%]av = J 2 /(^" 
1) for any i and j. This model, which is equivalent to 
an infinite dimensional model for .which mean field the- 
ory is exact, has a phase diagran£3 in (h/J,Jo/J,T/J) 
space which is reasonably well understood. In particu- 
lar, there are instability lines (which presumably form a 
surface) below which the replica symmetric solution is 
unstable, and the Parisi RSB ansatza is believed to give 
the correct solution. The 'AT line' found by de Almeida 
and Thoulesaij lies in the h-T plane (i.e., Jo = 0); be- 
low this AT line there is RSB, while increasing either h 
or T brings one to a phase consisting of a single pure 
state. This phase is continuous with the paramagnetic 
phase at h = 0; it has long-range order which is triv- 
ially induced by the field, and hence neither spontaneous 
symmetry breaking nor broken ergodicity of any other 
sort. Nevertheless we expect a large eigenvalue for C due 
to the long-range order. In another region of parameter 
space (h = 0, with a ferromagnetic bias Jo sufficiently 
large) there is a ferromagnetic phase with one PSP. Here 
one has the familiar version of broken ergodicity in the 
form of spontaneous symmetry breaking; given that there 
is a single PSP, we expect a single extensive eigenvalue 
in this phase also. Thus we find three distinct phases 
(glass/RSB, paramagnet -(-field, ferromagnet) which we 
can explore via Monte Carlo simulations in order to test 
our ideas about correlation functions, eigenvalue count- 
ing, and ergodicity breaking iaJsing systems. 

We have performed EMQL3lI simulations for the SK 
model at three points in the phase diagram. In glassy 
systems with very long relaxation times, normal Monte 
Carlo simulations are limited to small system sizes 
because, of the divergent relaxation and equilibration 
times.E3 EMC simulations allow for the crossing of barri- 
ers in a reasonable simulation time, via a stochastic walk 
of each simulated system not only in configuration space 
but also in temperature. The simulation consists of hav- 
ing many systems at different temperatures for the same 
disorder realization running in unison, and attempting 
to exchange the configurations between adjacent tem- 
peratures after a given number of Monte Carlo steps St. 
The exchange of neighboring temperature configurations 
takes place with the probability 



,-A 



with 



A = (An - (3 m+1 )(E(S m+1 ) - E(S m )) . 



(25) 



(26) 



and S m indicating the instantaneous spin configuration 
at temperature l//3 m . With this probability of exchange 
one ensures that the systems at the different temper- 
atures remain in thermal equilibrium — whether or not 
they exchange temperatures — at all times during the sim- 
ulation. This is because the configuration obtained from 
an exchange with a higher temperature is still accepted 
with the normal Boltzmann probability for the lower (ac- 
cepting) temperature; hence an exchange drives neither 



system out of equilibrium. If the temperature difference 
between neighboring systems is not too large, then ex- 
changes are accepted at a reasonable rate, and each sys- 
tem explores the full range of temperatures. Hence each 
system is effectively cooled and heated many times dur- 
ing the simulation, ensuring that all barriers have been 
effectively circumvented, not by crossing them but by 
falling within their boundaries from a higher tempera- 
ture. An important parameter in this simulation is the 
spacing between the different temperatures, which must 
be adjusted to get a high||6»ough acceptance ratio for the 
temperature exchanges .EtH3 

Our criteria for having reached thermal equilibrium in 
our measured quantities involve calculating the spin-glass 
susceptibility for a single disorder realization using two 
distinct methods-- .One method, discussed extensively in 
previous studies JHJ uses the averaging of the overlap of 
two uncoupled replicas 
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fo+^i (to + i)) 



(27) 



and the other uses the standard way of calculating the 
thermal average in Monte Carlo simulations. 



(2) 




Si(to+t)S s (t + t) 



(28) 



Here To and to have to be chosen large enough to ob- 
tain thermally equilibrated results. We also demand 
full symmetry of the overlap distribution function P(q), 
and that all initial configurations visit all temperatures 
evenly. We have also checked our results using commonly 
studied quantities such as P(q) and xsg doing standard 
Monte Carlo simulations for the smaller system S^B** 
here our results are in agreement with previous workESES 
In our simulations we have used a temperature spacing 
of AT = 0.05 J, t =3-5x 10 4 , r = 10 - 40 x 10 4 , and 
St = 10 — 20. (Here all times are in units of Monte Carlo 
steps per spin.) 

We find that the eigenvalue spectrum of C is strongly 
dependent on disorder realization, such that it is impos- 
sible to observe any regular dependence of the A, on N 
with increasing N for a single set of Jy's. (See Fig. ^ 
below for examples of similar behavior in four space di- 
mensions.) Hence it is necessary to accumulate statis- 
tics on the eigenvalue spectra for many disorder real- 
izations. The eigenvalue probability distribution for the 
first two eigenvalues of C,j for N — 128 at zero field 
and at h = 1.2J (above the AT line)— all at T = 0.4J— 
are shown in Fig. |l| These distributions have been ob- 
tained from 3400 disorder realizations. It is clear from 
Fig. |] that, at least in the RSB phase, the distributions 
for Ai and A2 are extremely broad; also they show sig- 
nificant skew. Hence we have studied both [Aj] av an d 
[Ai]ty P = exp[lnAi] av for small i in each phase. 

We show the average of the ten largest eigenvalues as 
a function of system size in Fig. || for different points of 
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the SK phase diagram: (a) h = 0, T/J = 0.4, J = 0, the 
RSB phase; (b) fr/J = 1.2, T/J = 0.4, J = 0, the para- 
magnet in field with one pure state; and (c) h = 0, TJ J = 
0.4, Jo/ J — 1.5, the ferromagnet with a single PSP. The 
system sizes considered are N = 32, 64, 128, 192, 256, 
and 512 with 3400, 3400, 3400, 1400, 1100, and 400 dis- 
order realizations performed respectively. It is clear from 
Fig. ||(a) that two eigenvalues are of order N for N > 200. 
This is extremely strongLJ numerical evidence for more 
than one PSP, and hence nontrivial ergodicity breaking. 
We expect further O(N) eigenvalues to emerge for larger 
N, as suggested by the behavior of [As] av in the figure. 

In contrast, there is only one large eigenvalue in Fig. 
||(b) and (c). We find further that [\2)a,v/N may be fit 
to a power law N~ S2 for some range of N in cases (b) 
and (c). In the paramagnetic phase (b) 62 ~ 0.52 while 
in the ferromagnet (c) 62 is somewhat larger, ~ 0.58. 
While we do not expect these power laws to have reached 
their asymptotic values for the system sizes considered 
here, we do expect any observed decay for large enough 
N to be consistent with our arguments in the previous 
section, where we obtained 6% > 1/2 in this regime. MC 
results for the paramagnetic phase, in a larger field than 
in (b), show that [\2]a,v/N decays with a larger exponent 
(we have observed up to ~ 0.75), which we expect to 
approach 1 for large enough h. We also show in Fig. 
H the scaling of the typical value [Aj]t yp /./V for the first 
ten eigenvalues in the RSB phase. Here the behavior 
is qualitatively similar to that of [Ai] av /7V in Fig. ||(a): 
both figures show clearly that [\2]/N (av or typ) is fiat 
as a function of N above a threshold value for N which 
is of order 100-200, plus strong signs that [A3] /TV and 
[A^/iV are also approaching a flat behavior. Hence we 
see graphically, in these figures, the emergence of multiple 
PSPs with increasing N. 

Figs. p|and|3[ taken together, give convincing evidence 
that the eigenvalue spectrum of C can clearly distinguish 
trivial (one PSP) from nontrivial broken ergodicity. This 
spectrum allows one, for large enough N, to detect mul- 
tiple PSPs simply by counting the number of extensive 
eigenvalues Neev- As discussed above, while the num- 
ber of PSPs is believed to diverge in the thermodynamic 
limit, only a few of them dominate the thermodynam- 
ics;^ we see some of these emerging in Figs. ||(a) and ||. 
A possible consequence of this would be that, for large 
enough N, Neev might saturate at a finite constant. 
The present data show no clear sign of this, although 
they are clearly beyond the threshold of N for which 
Neev begins to exceed 1. For N in the vicinity of this 
threshold, we expect that one can reasonably view the 
glass phase as having just two PSPs; and we explore the 
consequences of this assumption next. 



IV. TWO PURE STATE PAIRS 

As suggested by Figs. |l|, ||, and ||, the spectrum of C for 
the SK problem is dominated by two large eigenvalues, 
each coming from a broad distribution of values, for the 
system sizes considered here. We can understand much 
of this behavior with a simple two-PSP model which can 
be computed analytically. We begin with a very sim- 
ple model for two PSPs at T = 0. Suppose that phase 
space consists of only two spin configurations 1 and 2, 
and ignore all others. Take C(w) — wC° + (1 — w)C2, 
with C® and C 2 corresponding to the Cy of the two 
different configurations and at zero tempera- 
ture, and w (a thermodynamic weight) ranging from 
to 1/2. The overlap between the two states is given by 
<7i2 = J2i SiiS^i/N. It can be easily shown that this 
matrix has only two nonzero eigenvalues, corresponding 
to 



X±(qu,w) _ l±^q( 2 + (l-q( 2 )(2 W -iy 



N 



(29) 



Note that A + ranges from N at w = to (1 + |qi 2 |)iV/2 
at w = 1/2. It is also clear that, for small w, A_ is linear 
in w; hence if w is of lower order in N than TV , A_ ceases 
to be extensive. At the same time, even if the two PSPs 
have equal thermodynamic weight, the second eigenvalue 
A_ approaches zero as \qi2\ ~ ¥ 1 an d N/2 as Igioj — > 0. 
This is consistent with our results from Section [n], since 
as I Q12I — > 1 the two states become linearly dependent 
and p becomes 1. We can proceed further with this two- 
PSP model by calculating the probability distribution of 
Ai and A2, as follows: 



)PM , (30) 



P w (\i) = I dq Pi 2 (q)5(\i - X+(q,w)) 

A+ 



2Vq 2 + (l-qo) 2 (2w-iy 



? o(l-(2w-l) 2 ) 



0(Ai 



where P\2{q) is the probability distribution of q\2, X+ m in 
depends on w, and qo is an implicit function of Ai given 
by A + (<7o, w) = Ai. For the case of w = 1/2 this simplifies 
to 

jW/ 2 (Ai) = 2P 12 (2A 1 /7V - l)0(Ai - A +mm ) , (31) 
and 

JW/ 2 (A 2 ) = 2P 12 (1 - 2\ 2 /N)0(\- max - A 2 ) ; (32) 

here A+ m «n = X-max = 1/2- At this simple level of ap- 
proximation we already have the first indication that the 
probability distribution of the first and second eigenval- 
ues will be very broad in the case of non-trivial broken 
ergodicity — as we have seen in the MC results. Note 
however that in spite of the breadth of the distribution 
[A±] av are still proportional to N. Let us now augment 
this picture with finite-temperature effects. An approx- 
imate way to introduce temperature into the two-PSP 
model is as follows. We let 
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c 



(1),(2) 



o .(l),(2) o .(l),(2) 

4 J 



where — (jSj)W and a^' = (Si)^' are gaus- 

sian random variables with the mean of of equal to 
Imax- We adjust the distribution of and of' 

such that q m ax agrees with that obtained from our MC 
runs, while also demanding that the overlap distribution 

(1/N) J2i CT ! 1 '' cr ! 2 ^ agree with Pu(q) — which can also be 
extracted from our MC P(q), by subtracting a Gaussian 
part due to the self-overlap Pn(q) = ^22(9)- The result 
of this decomposition procedure is illustrated in Fig. ^. 
This gives us a means of generating a realistic sample of C 
matrices corresponding to two PSPs. We can then obtain 
the first few eigenvalues of CV, (w) — luC-j' + (1 — w)C^ 
and compare them with the eigenvalues obtained directly 
from the MC simulations of the SK model. The former 
eigenvalues may be obtained either by an approximate 
analytical perturbation approach (using only the diag- 
onal part of V), or by direct numerical diagonalization 
of the matrix C(w) — which must in any event be gen- 
erated by a random number generator. Fig. |E| shows 
the eigenvalue distributions obtained from our two-PSP 
model with w — 1/2, compared with those obtained di- 
rectly from the EMC runs, for the SK model in the glass 
phase at TV = 64. It is encouraging that our simple pic- 
ture of two PSPs, with a minimal set of assumptions, 
reproduces both the position and the shape of the two 
distributions. 

We believe — and Fig. || supports this belief — that the 
assumption of two PSPs has validity for a range of N 
near the threshold where RSB first appears. We also 
believe that this assumption will fail for larger N; our 
Monte-Carlo results strongly suggest that the number p 
of significant PSPs will exceed two as N grows. It is 
interesting to ask what the large-iV limit of p is. We 
obtain no answer to this question from the considerations 
of Section 0; while our numerical results suggest only 
that p is at least as large as 3 or 4. We note here that 
Fig. I itself may be viewed as giving some indication of a 
third large eigenvalue, if we assume (as is plausible from 
our two-PSP results) that the third PSP robs weight from 
the upper part of P(A2). 



(2) 



for i — j 
for i ^ j , 

\(2) 



(33) 



V. RESULTS FOR THE EA MODEL IN FOUR 
DIMENSIONS 

We have performed EMC studies of the four- 
dimensional Ising spin glass on a hypercubic lattice, with 
nearest-neighbor interactions, periodic boundary condi- 
tions, and a gaussian distribution of the Jij's with zero 
mean. The parameters of the calculation follow closely 
those of Ref. [| We have focused on the point T = J and 
h = 0. These simulations are rather deep in the frozen 
phase, since T c 1.75 J. We choose this low temperature 
in order to try to avoid spurious effects from closeness to 



the critical region; such effects are likely to make it diffi- 
cult to distinguish multiple PSPs from a single PSP. The 
price we pay is that our MC runs are slow to converge, 
while — according to the estimates of Ref. [l^ — we are still 
not fully out of the critical region. Our criteria for con- 
vergence are the same here as those we used for the SK 
model (Sec. EJ). 

We have only examined the frozen phase for this prob- 
lem. We find that a plot of the distributions for Ai and 
A2 gives broad and skewed forms similar to those seen 
for the RSB phase in Fig. [I]. To complement the pic- 
ture given in Fig. [j], we show in Fig. |^ some examples 
of the typical behavior of a single disorder realization at 
"fixed" Jij's and increasing N. Here "fixed" is in quo- 
tation marks since (as is well known for spin glasses) 
adding spins requires adding bonds, and hence a change 
in the { Jy}, which can often have nontrivial effects. Fig. 
^ bears out this expectation: the eigenvalue spectrum of 
C shows a highly irregular behavior as a function of N . 
If this irregularity were to persist in the limit N — > 00 
(such that the eigenvalues, and hence the correlations, 
had no well-defined limit), then according to Newman 
and Stein,E3 there must be more than a single pure-state 
pair. That is, "chaotic size dependence" is believed to 
characterize glasses with RSB, but not to occur fcp,a 
single PSP (unlike chaotic temperature dependenceEa) . 
While the behavior shown in Fig. ^| is interesting in this 
regard, we do not believe any conclusion can be drawn 
from these data due to the small size of the systems con- 
sidered here. Instead we will focus on trying to count 
extensive eigenvalues — a strategy that worked well for 
the SK problem. Fig. ^ suggests rather strongly that Ai 
is extensive; but it is impossible to draw any conclusion 
about A2, and so we again resort to disorder averaging. 

In Fig. [?] we show (a) [Aj] av /7V and (b) [Ai]t yp /iV for 
the first ten eigenvalues of C, at T = J and h = 0. 
The system sizes shown in Fig. [?] are N = 81, 256, 625, 
and 1256 with 4000, 1600, 1300, and 400 disorder real- 
izations respectively. These data suggest that [\2\&v/N 
and [A2]t yp /-/V are each decaying with N, with a clean 
power law N~ S2 . A fit of the data in Fig. [?] gives (a) 
for the average eigenvalues, <5f v ~ 0.11 and (b) for the 
typical eigenvalues, S^ p ~ 0.15. The exponent we find 
for \\2]tw /N is close to that expected from our argument 
of Section II A| , coupled with^Eeyious estimates for the 
exponent 9. The latter_rangeE3't3 from 0.6 to 0.8, while 
a simple extrapolation^] suggests 8 ~ 0.7. Our own ar- 
gument (<$2 > 0/4) predicts a minimum value for 5% in 
the range 0.15 to 0.2; hence the behavior of [AaJtyp/iV 
is roughly consistent with this prediction, while <5| v for 
[A2]av/-^V is somewhat smaller. 

The decay of [A2]/iV is quite regular. Moreover, there 
is rough agreement between the exponent obtained from 
the typical eigenvalues and the exponent estimated by 
assuming the droplet picture to be valid. This evidence 
seems to favor the scenario of only one PSP being present. 

Of course, it is in general difficult to settle from nu- 
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merical simulations alone any question that involves be- 
haviors of a physical system in the thermodynamic limit, 
since the results observed for some sizes can change when 
larger sizes are simulated. In particular, any claim that 
RSB does not occur, based on our method, is necessarily 
more tentative than a conclusion that it does occur.EEl 
Thus the results we obtain for N < 1296 cannot rule out 
the possibility of further PSPs appearing at some larger 
N, indicating that there is RSB in the spin-glass phase 
in four dimensions. By contrast, if the droplet picture is 
correct for 4D spin glasses, then we believe that the de- 
cay exponent 82 should, for sufficiently large N, increase 
from its lower bound as the dominance of a single droplet 
fails. Hence any sign of curvature of the log- log plot of 
X2/N vs. N, in either direction, would be of significant 
interest. 

It is also of interest to push results like those of Fig. ^ to 
larger N. Here one seeks signs of convergence (or lack of 
it) to a limit. This criterion is, we believe, more difficult 
to assess than the criterion we have applied to Figs. |[ [|, 
and 0. The latter criterion has the nice property that one 
must only ascertain whether an integer — the number of 
extensive eigenvalues — is one, or greater than one. How- 
ever studies seeking chaotic size dependence can certainly 
complement studies of disorder-averaged eigenvalue scal- 
ing. 

VI. CONCLUSIONS 

In this work we have applied the old idea of study- 
ing the eigenvalue spectrum of a correlation function — 
used by YangL3 to characterize ODLRO in superfluids — 
to a decades-old question in spin-glass physics, namely: 
how many pure states are there in the frozen phase, and 
how are they related? The connection we have made 
is simple: for problems in which the low-temperature 
phase has multiple pure states (of non-negligible ther- 
modynamic weight) not related by spin inversion sym- 
metry, the broken ergodicity shows up as multiple ex- 
tensive eigenvalues of the spin-spin correlation matrix 
C. We have strong arguments in two directions: first, 
that the presence of multiple extensive eigenvalues nec- 
essarily implies nontrivial ergodicity breaking, i.e., mul- 
tiple pure-state pairs; and second, that the presence of 
multiple pure-state pairs will give rise to multiple ex- 
tensive eigenvalues. We have found striking support for 
these arguments from numerical (Monte-Carlo) studies 
of the Sherrington-Kirkpatrick problem in three distinct 
phases — the paramagnetic, ferromagnetic, and replica- 
symmetry-broken (RSB) phases. Specifically, we find 
clear and unambiguous signs of the different kinds of 
ergodicity breaking in these three phases via studies of 
the N dependence of the disorder-averaged eigenvalues 
of C — which essentially enable us to count the number 
of extensive eigenvalues, and hence the number of pure- 
state pairs in the configuration space. We believe that 



the evidence for RSB displayed in Figs. ||(a) and || is 
unique in its directness, clarity and lack of ambiguity. 

We have also applied these ideas to the near-neighbour 
Ising spin glass in four dimensions. Our data are consis- 
tent with the presence of only one extensive eigenvalue, 
for N < 1296. Furthermore, the typical value of X2/N 
decays with a clean power law; and the exponent agrees 
roughly with the value expected from an argument based 
on the droplet picture, plus independent estimates of the 
scaling exponent 9. An alternative analysis that assumes 
that a second extensive eigenvalue is present with large 
finite size corrections cannot be completely excluded, al- 
though the lack of any restriction on the fitting parame- 
ters makes any conclusions drawn from such fits (in gen- 
eral of higher x 2 ) questionable. Thus our results tend 
to support the 'droplet' picture of the frozen phase, with 
a single PSP, more strongly than they support the RSB 
picture. We believe that studies of the kind reported 
here should be extended to larger N in order to test this 
tentative conclusion. Our present results encourage us 
to believe that such studies can play an important role 
in settling the question, from the theoretical side, of the 
nature of the broken ergodicity in real spin glasses. 

The authors acknowledge helpful discussions with J. 
Hu, E. Sorensen, and G. Parisi. This work was supported 
by the National Science Foundation under grants DMR- 
9820816 and DMR-9714055. 



APPENDIX A: PROPERTIES OF THE 
MATRICES C°, V AND A 

In this appendix we show that: (i) the symmetric ma- 
trices defined in Sec. |l| C°, V (both of size N x N) and 
A (of size p x p), are positive semi-definite, (ii) the rank 
of A is equal to p (the number of pure states with linearly 
independent magnetizations), (iii) the eigenvectors of C° 
constructed via Eq. ([l8]) from the linearly independent 
eigenvectors of A with positive eigenvalue are linearly 
independent, and (iv) the remaining N — p linearly inde- 
pendent eigenvectors of C° have zero eigenvalue and are 
orthogonal to all pure state magnetizations. 

We start by showing that C° is positive semi-definite: 
for an arbitrary real N-dimensional vector \v), 

v 

E v i C ij v i = E v i v o E w a mfmf 

ij ij a— 1 

= I> a (E^<) 2 ^ - ( A1 ) 

a— 1 i 

Similarly, in the case of V we have, 

ij 

= E E WW* <^r)r 

ij a—1 
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= i> a <(£^ ~ (Si) a ))V > 0. (A2) 

a— 1 z 

We now concentrate in studying the p x p matrix A. 
By convention, we enumerate the pure states so that the 



£(y»)c 



(A9) 



a=l 



first p magnetization vectors {m"} Q= 



are linearly 



by Eq. (|Aj). Combining this with Eq. (|A3|) it follows 
that for an arbitrary p dimensional real vector |x) 



independent. Now, for any real p-dimensional vectors 
I a;) and \y) we have, 



(x\A\y) = 



{x\A\y a 



0. 



(A10) 




(A3) 



This means that all of the \y s ) are eigenvectors of A with 
zero eigenvalue. By the Gram-Schmidt procedure, an 
orthonormal set {|up + i), • • • , \v p }} of linear combinations 
of th em c an be constructed. By combining Eq. (A6) and 
Eq. JA10|) we get: 



By choosing \x) = \y) = \v) wc immediately see that A 
is positive semi-definite. Thus statement (i) is proven. 

We will now study the eigenvectors of the matrix A. 
Consider Eq. ( |A3| ) in the case that \x) — \y) — \v) and 
\v) is chosen such that 



= , p < a < p 



p 

£ 

a=l 



{v a f > 



(A4) 



Since by our assumptions {m'j} a= i r .. t p are linearly inde- 
pendent and {w a } Q =i,- -,p are nonzero, we have that 



p 

a=l 



(v\A\v) > 0. 



(A5) 



By the Gram-Schmidt procedure, we can obtain p nor- 
malised vectors {vA that are orthogonal with respect to 
A, all satisfying (A4). This means that 



•\A\vs 



(A6) 



where the {a r }r=i, --,p are positive numbers. The {v r } 
will be shown below to be eigenvectors of A with eigen- 
values a r > 0, but first it is necessary to construct the 
eigenvectors with zero eigenvalue. To do that, let us con- 
sider the magnetization vectors m" with p < a < p, that 
can be written as linear combinations of the first p ones 
(i.e. the ones that are linearly independent): 



0=1 



(A7) 



This allows to construct the p — p linearly independent 
vectors \y s ), s = p + 1, • • • ,p of the form 




, for a = 1,---,P, 
for a = p + 1, • • • ,p, 



(A8) 



A\v r 



\v r ) for r = 1, • 

> for 1 < r < p, 
= for p < r < p, 



,p, 



(All) 
(A12) 



and therefore statement (ii) is proven. 

Consider now the vectors of the form proposed in 
Eq. ( |l8| ) associated with the p eigenvectors of A with posi- 
tive eigenvalue. Since the matrix (v r ) a (r, a = 1, • • • ,p) is 
invertible, and the magnetization vectors {m a } a =i t ... t p) 
are linearly independent, it follows that the vectors 
{<pr}r=i,---,p are linearly independent. Thus statement 
(iii) is proven. 

The remaining N — p eigenvectors of C° can be con- 
structed as follows. Take an arbitrary vector \u) in the 
(N — p)-dimensional subspace orthogonal to the magne- 
tization vectors {m a } a =i,- -,p (and consequently orthog- 
onal to all of the p magnetization vectors of the pure 
states). Clearly, \u) is a null vector for C° since 



E 



W 171, 



= 0, 



(A13) 



and this proves statement (iv). 

APPENDIX B: SPECTRUM OF C° + V 

In this appendix some bounds are shown to be satis- 
fied by the changes in the eigenvalues of the spin-spin 
correlation matrix due to the effect of V. 

For a given disorder realization, temperature T and 
size N, let us consider the eigenvalues and eigenvectors 
of the correlation matrices C = C° + V and C°: 



C\Xr) = Xr\Xr), 
C°|0 r ) =K r \4> r ), 



(Bl) 



where the eigenvalues are positive and labeled in descend- 
ing order (the subscript 1 corresponding to the highest 
eigenvalue for each matrix). We label by v the largest 
eigenvalue of the matrix V. We will use variational ar- 
guments to show that: 
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• For any value of p, 

Ki + v > A r > n r for r = 1, • • • ,p. (B2) 

• For p = 1, and if the ratio v/ki is small enough, 
then: 



v 



v 



x 2 < v (i + — + o((—y)). 

Ki Kl 



(B3) 



Let's consider Eq. (B2) first. The upper bound is ob- 
vious once C is decomposed as a sum of C° and V, and 
use is made of the fact that any mean value for each one 
of them has to be smaller or equal than their respective 
maximum eigenvalues: 

A,- = (Xr\C\ X r) = (Xr\C°\ X r) + (Xr\V\ X r) < «1 + V 

(B4) 

To prove the lower bound in the case r = 1, we just 
consider the eigenvector \cj)i) corresponding to the largest 
eigenvalue Ki of C , and use it as a variational trial vec- 
tor: 



A 1 >(^i|C|^i) = /si + <0i|V|0i)>/si 1 



(B5) 



where we have used the fact that V is positive semi- 
definite. 

To prove the lower bound for general r = 1, ■ ■ ■ , j>, we 
use an inductive reasoning. We assume that Eq. ( jB2| ) is 
valid for all r' — 1, ■ • • , r — 1, and propose as a variational 
trial vector a linear combination \x) of \<j>\ ),•••, | </>?•)• Be- 
cause it is generated by r linearly independent vectors, 
this linear combination can be chosen to be orthogonal to 
all of the r — 1 exact eigenvectors |xi)> * * * j IXr-0- Then 
we have 



K>(x\c\x) = (x\c v \x) + (x\v\x)>^ 



(B6) 



where we have used the facts that i) the term correspond- 
ing to C° has lower bound, and ii) V is positive 
semi-defi nitc . This proves the inductive step, and there- 
fore Eq. (pi. 



Let's consider the inequality Eq. (B3). To prove it, we 
decompose the eigenvector \x\) into a part proportional 
to \4>\) and a part orthogonal to it: 



1x0 = cos(-)|^i> + sm(-)|77>, 



(B7) 



where \r)) is a normalised vector orthogonal to \<j>i). We 
now write the matrix elements of C in the subspace gen- 
erated by |x0 and \rj): 

V (mv\(f>i) (v\v\m J 



The coefficient 9 that parametrizes Eq. (B7) can be re- 
lated to these matrix elements by 



tan ft* = 



2{4n\V\ri) 



K 1 + {<h.\V\th)-{v\V\vy 
and therefore satisfies the bound 

2v 



tan 61 < 



Kl — V 



(B9) 



(BIO) 



We now consider the eigenvector \x2), corresponding to 
A2, the second largest eigenvalue of C. Since is nor- 
malized and orthogonal to |xi), it has the form: 



IX2) - cos(|)(- sin(-)|0 1 ) + oos(-)|T7» + MpW), 

(BH) 

where \rj') is a normalized vector orthogonal both to |xi) 
and to 1 77). From the expression for A2: 

A 2 = (X2ICIX2) = (X2\C°\ X 2) + (X2\V\X2) 

= cos 2 (^)sin 2 (^)(0 1 |C K) |0 1 ) + (x2^|X2) (B12) 



we immediately obtain the bound: 
A 2 < sin (-)ki + v. 



(B13) 



From Eq. ( B1C ), it is clear that for v -C ki we have: 

n 2 



^ 2 Q 



tan 9 



(1 + 0(9*)) 



Kl Kl 



(B14) 



which combined with Eq. (B13) implies Eq. (B3) 
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FIG. 1. Distribution of the first (thick lines) and second 
(thin lines) eigenvalues of dj in the SK model at h/J = 
(RSB phase; solid line) and h/J — 1.2 (paramagnetic phase; 
dashed line). Here T/J = 0.4 and N = 128. These distri- 
butions, and the ones shown in Fig. ^, are a smoothed fit to 
binned data. 
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FIG. 3. Scaling of the typical value ([Ai] typ = exp[lnA;]av) 
of the ten largest eigenvalues of dj as a function of system 
size N in the SK model below the AT line (h = 0, T/J = 0.4). 
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FIG. 2. Scaling of the disorder average of the ten largest 
eigenvalues of dj as a function of system size N in the SK 
model (a) below the AT line (h = 0, T/J = 0.4), (b) above the 
AT line (h/J = 1.2, T/J = 0.4), and (c) in the ferromagnetic 
phase {h = 0, T/J = 0.4, [Jij] m = 1.5J). The error bars in 
this and subsequent figures come from the disorder averaging. 



4.00 



3.00 



P(q) from SK model at T=0.4 
P u (q) from simulation 
P l2 (q) from simulation 
P(q) from simulation 




1.00 



FIG. 4. Overlap distribution function obtained from MC 
calculations in the SK model at T = 0.4 and h — (filled cir- 
cles), self-overlap distribution function Pu(q) (open squares), 
mutual overlap distribution function P12 (q) (open triangles) , 
and the total P(q) used in the two-PSP model simulations 
(solid line). 
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FIG. 5. Distribution of the first (solid line) and second 
(dashed line) eigenvalues of CV,- obtained from the MC simu- 
lation of the SK model at N = 64 and T/J = 0.4 (thin line), 
and the respective distributions obtained from the two pure 
state model simulation (thick line) with w = 1/2. 
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FIG. 7. Scaling of the average (a) and the typical (b) value 
of the ten largest eigenvalues of dj as a function of system 
size N for h = and T/J = 1.0 for the EA model in four 
dimensions. 
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FIG. 6. Four examples of the behavior of the {Xi/N}, 
1 < i < 5, for a single disorder realization but increasing 
N. Error bars are from Monte-Carlo fluctuations rather than 
disorder averaging; dashed lines connecting Ai(iV) points, and 
\2(N) points, are simply guides to the eye. We see that the 
behavior is very irregular, allowing no conclusions about the 
number of extensive eigenvalues. 
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